<template><div><h2 id="知识点" tabindex="-1"><a class="header-anchor" href="#知识点"><span>知识点</span></a></h2>
<h3 id="webrtc、rtsp、rtmp的协议栈分别长什么样-区别是什么" tabindex="-1"><a class="header-anchor" href="#webrtc、rtsp、rtmp的协议栈分别长什么样-区别是什么"><span>WebRTC、RTSP、RTMP的协议栈分别长什么样？区别是什么？</span></a></h3>
<p><strong>WebRTC</strong></p>
<blockquote>
<p>what: Web实时通信协议,基于js原生api实现,支持客户端直接通信</p>
</blockquote>
<blockquote>
<p>where: 一般用来处理视频会议,实时互动直播,P2P文件传输</p>
</blockquote>
<blockquote>
<p>why: 基于UDP协议进行通信,保证实时传输</p>
</blockquote>
<p><strong>协议栈</strong></p>
<div class="language-cpp line-numbers-mode" data-highlighter="shiki" data-ext="cpp" style="background-color:#282c34;color:#abb2bf"><pre class="shiki one-dark-pro vp-code" v-pre=""><code><span class="line"><span style="color:#ABB2BF">应用</span></span>
<span class="line"><span style="color:#ABB2BF">├─ 信令层：无固定协议（通常用WebSocket</span><span style="color:#C678DD">/</span><span style="color:#ABB2BF">HTTP，负责协商参数：编解码、网络地址等）</span></span>
<span class="line"><span style="color:#ABB2BF">├─ 媒体引擎层：</span></span>
<span class="line"><span style="color:#ABB2BF">│  ├─ 媒体协商：SDP（描述音视频参数：编码格式、带宽等）</span></span>
<span class="line"><span style="color:#ABB2BF">│  ├─ 加密传输：SRTP（音视频数据加密）、DTLS（密钥协商，替代SSL）</span></span>
<span class="line"><span style="color:#ABB2BF">│  └─ 媒体传输：RTP（实时媒体封装）</span><span style="color:#C678DD">+</span><span style="color:#ABB2BF"> RTCP（质量反馈：丢包率、延迟，动态调整码率）</span></span>
<span class="line"><span style="color:#ABB2BF">├─ 网络适配层：</span></span>
<span class="line"><span style="color:#ABB2BF">│  ├─ 穿透NAT：ICE（整合STUN和TURN，解决P2P连接问题）</span></span>
<span class="line"><span style="color:#ABB2BF">│  │  ├─ STUN：获取公网地址（无服务器转发）</span></span>
<span class="line"><span style="color:#ABB2BF">│  │  └─ TURN：穿透失败时通过服务器转发（兜底方案）</span></span>
<span class="line"><span style="color:#ABB2BF">│  └─ 传输层：UDP（默认，低延迟），可选TCP（丢包敏感场景，延迟略高）</span></span>
<span class="line"><span style="color:#ABB2BF">└─ 编解码层：</span></span>
<span class="line"><span style="color:#ABB2BF">   ├─ 视频：VP8</span><span style="color:#C678DD">/</span><span style="color:#ABB2BF">VP9（开源）、H.</span><span style="color:#D19A66">264</span><span style="color:#ABB2BF">（兼容）、AV1（新一代低带宽）</span></span>
<span class="line"><span style="color:#ABB2BF">   └─ 音频：OPUS（自适应带宽，兼顾音质和抗丢包）、G.</span><span style="color:#D19A66">711</span><span style="color:#ABB2BF">（传统语音，兼容性）</span></span></code></pre>
<div class="line-numbers" aria-hidden="true" style="counter-reset:line-number 0"><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div></div></div><blockquote>
<p>demo: <a href="https://blog.csdn.net/Python_0011/article/details/131936099#:~:text=%E6%9C%AC%E6%96%87%E4%BB%8B%E7%BB%8D%E4%BA%86WebRTC%EE%80%80%E7%9A%84%E5%9F%BA%EE%80%81%E6%9C%AC%EE%80%80%E6%A6%82%E5%BF%B5%E3%80%81%E4%BD%BF%EE%80%81%E7%94%A8%E7%A4%BA%E4%BE%8B%EE%80%80%E3%80%81%EE%80%81%E6%8A%80%E6%9C%AF%E8%83%8C%E6%99%AF%E5%92%8C%E5%8F%91%E5%B1%95%E5%8E%86%E7%A8%8B%EF%BC%8C%E4%BB%A5%E5%8F%8A%E5%A6%82%E4%BD%95%E9%80%9A%E8%BF%87RTCPeerConnection%E5%92%8CSDP%E8%BF%9B%E8%A1%8C%E9%80%9A%E4%BF%A1%E3%80%82%E6%96%87%E7%AB%A0%E8%BF%98%E6%8E%A2%E8%AE%A8%E4%BA%86NAT%E7%A9%BF%E9%80%8F%E9%97%AE%E9%A2%98%EF%BC%8C%E8%A7%A3%E9%87%8A%E4%BA%86STUN%E5%92%8CTURN%E7%9A%84%E4%BD%9C%E7%94%A8%EF%BC%8C%E5%B9%B6%E6%8F%90%E4%BE%9B%E4%BA%86%E4%B8%80%E6%AE%B5%E7%AE%80%E5%8D%95%E7%9A%84%E4%BB%A3%E7%A0%81%E7%A4%BA%E4%BE%8B%E6%9D%A5%E6%BC%94%E7%A4%BAWebRTC%E7%9A%84%E5%AE%9E%E7%8E%B0%E8%BF%87%E7%A8%8B%E3%80%82" target="_blank" rel="noopener noreferrer">WebRTC 入门与实践（非常详细）从零基础入门到精通，看完这一篇就够了_webrtc基本使用-CSDN博客</a></p>
</blockquote>
<div class="language-html line-numbers-mode has-collapsed-lines collapsed" data-highlighter="shiki" data-ext="html" style="--vp-collapsed-lines:15;background-color:#282c34;color:#abb2bf"><pre class="shiki one-dark-pro vp-code" v-pre=""><code><span class="line"><span style="color:#ABB2BF">&#x3C;!</span><span style="color:#E06C75">DOCTYPE</span><span style="color:#D19A66"> html</span><span style="color:#ABB2BF">></span></span>
<span class="line"><span style="color:#ABB2BF">&#x3C;</span><span style="color:#E06C75">html</span><span style="color:#D19A66"> lang</span><span style="color:#ABB2BF">=</span><span style="color:#98C379">"en"</span><span style="color:#ABB2BF">></span></span>
<span class="line"></span>
<span class="line"><span style="color:#ABB2BF">&#x3C;</span><span style="color:#E06C75">head</span><span style="color:#ABB2BF">></span></span>
<span class="line"><span style="color:#ABB2BF">    &#x3C;</span><span style="color:#E06C75">meta</span><span style="color:#D19A66"> charset</span><span style="color:#ABB2BF">=</span><span style="color:#98C379">"UTF-8"</span><span style="color:#ABB2BF">></span></span>
<span class="line"><span style="color:#ABB2BF">    &#x3C;</span><span style="color:#E06C75">meta</span><span style="color:#D19A66"> name</span><span style="color:#ABB2BF">=</span><span style="color:#98C379">"viewport"</span><span style="color:#D19A66"> content</span><span style="color:#ABB2BF">=</span><span style="color:#98C379">"width=device-width, initial-scale=1.0"</span><span style="color:#ABB2BF">></span></span>
<span class="line"><span style="color:#ABB2BF">    &#x3C;</span><span style="color:#E06C75">title</span><span style="color:#ABB2BF">>WebRTCDemo&#x3C;/</span><span style="color:#E06C75">title</span><span style="color:#ABB2BF">></span></span>
<span class="line"><span style="color:#ABB2BF">&#x3C;/</span><span style="color:#E06C75">head</span><span style="color:#ABB2BF">></span></span>
<span class="line"></span>
<span class="line"><span style="color:#ABB2BF">&#x3C;</span><span style="color:#E06C75">body</span><span style="color:#ABB2BF">></span></span>
<span class="line"><span style="color:#ABB2BF">    &#x3C;</span><span style="color:#E06C75">div</span><span style="color:#D19A66"> id</span><span style="color:#ABB2BF">=</span><span style="color:#98C379">"app"</span><span style="color:#ABB2BF">></span></span>
<span class="line"><span style="color:#ABB2BF">        &#x3C;</span><span style="color:#E06C75">video</span><span style="color:#D19A66"> id</span><span style="color:#ABB2BF">=</span><span style="color:#98C379">"localVideo"</span><span style="color:#D19A66"> controls</span><span style="color:#D19A66"> autoplay</span><span style="color:#ABB2BF">>&#x3C;/</span><span style="color:#E06C75">video</span><span style="color:#ABB2BF">></span></span>
<span class="line"><span style="color:#ABB2BF">        &#x3C;</span><span style="color:#E06C75">video</span><span style="color:#D19A66"> id</span><span style="color:#ABB2BF">=</span><span style="color:#98C379">"remoteVideo"</span><span style="color:#D19A66"> controls</span><span style="color:#D19A66"> autoplay</span><span style="color:#ABB2BF">>&#x3C;/</span><span style="color:#E06C75">video</span><span style="color:#ABB2BF">></span></span>
<span class="line"><span style="color:#ABB2BF">    &#x3C;/</span><span style="color:#E06C75">div</span><span style="color:#ABB2BF">></span></span>
<span class="line"><span style="color:#ABB2BF">    &#x3C;</span><span style="color:#E06C75">script</span><span style="color:#ABB2BF">></span></span>
<span class="line"><span style="color:#C678DD">        const</span><span style="color:#E5C07B"> pc1</span><span style="color:#56B6C2"> =</span><span style="color:#C678DD"> new</span><span style="color:#61AFEF"> RTCPeerConnection</span><span style="color:#ABB2BF">(); </span><span style="color:#7F848E;font-style:italic">// 本机端</span></span>
<span class="line"><span style="color:#C678DD">        const</span><span style="color:#E5C07B"> pc2</span><span style="color:#56B6C2"> =</span><span style="color:#C678DD"> new</span><span style="color:#61AFEF"> RTCPeerConnection</span><span style="color:#ABB2BF">(); </span><span style="color:#7F848E;font-style:italic">// 因为没有信令服务器,所以只能把本地端当作对端</span></span>
<span class="line"><span style="color:#ABB2BF">        </span></span>
<span class="line"><span style="color:#7F848E;font-style:italic">        // 监听ice候选者,并发送地址</span></span>
<span class="line"><span style="color:#E5C07B">        pc1</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">addEventListener</span><span style="color:#ABB2BF">(</span><span style="color:#98C379">'icecandidate'</span><span style="color:#ABB2BF">, </span><span style="color:#C678DD">async</span><span style="color:#ABB2BF"> (</span><span style="color:#E06C75;font-style:italic">e</span><span style="color:#ABB2BF">) </span><span style="color:#C678DD">=></span><span style="color:#ABB2BF"> {</span></span>
<span class="line"><span style="color:#7F848E;font-style:italic">            // 发送给对端</span></span>
<span class="line"><span style="color:#7F848E;font-style:italic">            // 对端添加本端地址</span></span>
<span class="line"><span style="color:#C678DD">            if</span><span style="color:#ABB2BF"> (</span><span style="color:#E5C07B">e</span><span style="color:#ABB2BF">.</span><span style="color:#E06C75">candidate</span><span style="color:#ABB2BF">) </span><span style="color:#C678DD">await</span><span style="color:#E5C07B"> pc2</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">addIceCandidate</span><span style="color:#ABB2BF">(</span><span style="color:#E5C07B">e</span><span style="color:#ABB2BF">.</span><span style="color:#E06C75">candidate</span><span style="color:#ABB2BF">);</span></span>
<span class="line"><span style="color:#ABB2BF">        });</span></span>
<span class="line"><span style="color:#ABB2BF">        </span></span>
<span class="line"><span style="color:#7F848E;font-style:italic">        // 监听连接事件</span></span>
<span class="line"><span style="color:#E5C07B">        pc1</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">addEventListener</span><span style="color:#ABB2BF">(</span><span style="color:#98C379">'iceconnectionstatechange'</span><span style="color:#ABB2BF">, (</span><span style="color:#E06C75;font-style:italic">e</span><span style="color:#ABB2BF">) </span><span style="color:#C678DD">=></span><span style="color:#ABB2BF"> {</span></span>
<span class="line"><span style="color:#E5C07B">            console</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">log</span><span style="color:#ABB2BF">(</span><span style="color:#98C379">'pc1: iceconnectionstatechange'</span><span style="color:#ABB2BF">, </span><span style="color:#E06C75">e</span><span style="color:#ABB2BF">);</span></span>
<span class="line"><span style="color:#ABB2BF">        });</span></span>
<span class="line"></span>
<span class="line"><span style="color:#E5C07B">        pc2</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">addEventListener</span><span style="color:#ABB2BF">(</span><span style="color:#98C379">'icecandidate'</span><span style="color:#ABB2BF">, </span><span style="color:#C678DD">async</span><span style="color:#ABB2BF"> (</span><span style="color:#E06C75;font-style:italic">e</span><span style="color:#ABB2BF">) </span><span style="color:#C678DD">=></span><span style="color:#ABB2BF"> {</span></span>
<span class="line"><span style="color:#7F848E;font-style:italic">            // 发送给本端</span></span>
<span class="line"><span style="color:#7F848E;font-style:italic">            // 对端添加本端地址</span></span>
<span class="line"><span style="color:#C678DD">            if</span><span style="color:#ABB2BF"> (</span><span style="color:#E5C07B">e</span><span style="color:#ABB2BF">.</span><span style="color:#E06C75">candidate</span><span style="color:#ABB2BF">) </span><span style="color:#C678DD">await</span><span style="color:#E5C07B"> pc1</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">addIceCandidate</span><span style="color:#ABB2BF">(</span><span style="color:#E5C07B">e</span><span style="color:#ABB2BF">.</span><span style="color:#E06C75">candidate</span><span style="color:#ABB2BF">);</span></span>
<span class="line"><span style="color:#ABB2BF">        });</span></span>
<span class="line"></span>
<span class="line"><span style="color:#E5C07B">        pc2</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">addEventListener</span><span style="color:#ABB2BF">(</span><span style="color:#98C379">'iceconnectionstatechange'</span><span style="color:#ABB2BF">, (</span><span style="color:#E06C75;font-style:italic">e</span><span style="color:#ABB2BF">) </span><span style="color:#C678DD">=></span><span style="color:#ABB2BF"> {</span></span>
<span class="line"><span style="color:#E5C07B">            console</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">log</span><span style="color:#ABB2BF">(</span><span style="color:#98C379">'pc2: iceconnectionstatechange'</span><span style="color:#ABB2BF">, </span><span style="color:#E06C75">e</span><span style="color:#ABB2BF">);</span></span>
<span class="line"><span style="color:#ABB2BF">        });</span></span>
<span class="line"></span>
<span class="line"><span style="color:#7F848E;font-style:italic">        // 对端监听轨道事件</span></span>
<span class="line"><span style="color:#E5C07B">        pc2</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">addEventListener</span><span style="color:#ABB2BF">(</span><span style="color:#98C379">'track'</span><span style="color:#ABB2BF">, (</span><span style="color:#E06C75;font-style:italic">e</span><span style="color:#ABB2BF">) </span><span style="color:#C678DD">=></span><span style="color:#ABB2BF"> {</span></span>
<span class="line"><span style="color:#C678DD">            if</span><span style="color:#ABB2BF"> (</span><span style="color:#E5C07B">e</span><span style="color:#ABB2BF">.</span><span style="color:#E5C07B">streams</span><span style="color:#ABB2BF">.</span><span style="color:#E06C75">length</span><span style="color:#56B6C2"> ></span><span style="color:#D19A66"> 0</span><span style="color:#ABB2BF">) {</span></span>
<span class="line"><span style="color:#E5C07B">                remoteVideo</span><span style="color:#ABB2BF">.</span><span style="color:#E06C75">srcObject</span><span style="color:#56B6C2"> =</span><span style="color:#E5C07B"> e</span><span style="color:#ABB2BF">.</span><span style="color:#E06C75">streams</span><span style="color:#ABB2BF">[</span><span style="color:#D19A66">0</span><span style="color:#ABB2BF">]; </span><span style="color:#7F848E;font-style:italic">// 播放视频</span></span>
<span class="line"><span style="color:#ABB2BF">            }</span></span>
<span class="line"><span style="color:#ABB2BF">        });</span></span>
<span class="line"></span>
<span class="line"><span style="color:#7F848E;font-style:italic">        // 获取对应元素</span></span>
<span class="line"><span style="color:#C678DD">        const</span><span style="color:#E5C07B"> remoteVideo</span><span style="color:#56B6C2"> =</span><span style="color:#E5C07B"> document</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">querySelector</span><span style="color:#ABB2BF">(</span><span style="color:#98C379">'#remoteVideo'</span><span style="color:#ABB2BF">);</span></span>
<span class="line"><span style="color:#7F848E;font-style:italic">        // 获取对应元素</span></span>
<span class="line"><span style="color:#C678DD">        const</span><span style="color:#E5C07B"> localVideo</span><span style="color:#56B6C2"> =</span><span style="color:#E5C07B"> document</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">querySelector</span><span style="color:#ABB2BF">(</span><span style="color:#98C379">'#localVideo'</span><span style="color:#ABB2BF">);</span></span>
<span class="line"></span>
<span class="line"><span style="color:#7F848E;font-style:italic">        // 推流</span></span>
<span class="line"><span style="color:#C678DD">        async</span><span style="color:#C678DD"> function</span><span style="color:#61AFEF"> pushStream</span><span style="color:#ABB2BF">(</span><span style="color:#E06C75;font-style:italic">answer</span><span style="color:#ABB2BF">) {</span></span>
<span class="line"><span style="color:#E5C07B">            pc1</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">setRemoteDescription</span><span style="color:#ABB2BF">(</span><span style="color:#E06C75">answer</span><span style="color:#ABB2BF">);</span></span>
<span class="line"><span style="color:#ABB2BF">        }</span></span>
<span class="line"></span>
<span class="line"><span style="color:#7F848E;font-style:italic">        // 拉流</span></span>
<span class="line"><span style="color:#C678DD">        async</span><span style="color:#C678DD"> function</span><span style="color:#61AFEF"> pullStream</span><span style="color:#ABB2BF">(</span><span style="color:#E06C75;font-style:italic">offer</span><span style="color:#ABB2BF">) {</span></span>
<span class="line"><span style="color:#E5C07B">            pc2</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">setRemoteDescription</span><span style="color:#ABB2BF">(</span><span style="color:#E06C75">offer</span><span style="color:#ABB2BF">); </span><span style="color:#7F848E;font-style:italic">// 交换sdp</span></span>
<span class="line"><span style="color:#C678DD">            const</span><span style="color:#E5C07B"> answer</span><span style="color:#56B6C2"> =</span><span style="color:#C678DD"> await</span><span style="color:#E5C07B"> pc2</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">createAnswer</span><span style="color:#ABB2BF">();</span></span>
<span class="line"><span style="color:#E5C07B">            pc2</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">setLocalDescription</span><span style="color:#ABB2BF">(</span><span style="color:#E06C75">answer</span><span style="color:#ABB2BF">);</span></span>
<span class="line"><span style="color:#E5C07B">            console</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">warn</span><span style="color:#ABB2BF">(</span><span style="color:#98C379">'answer'</span><span style="color:#ABB2BF">, </span><span style="color:#E06C75">answer</span><span style="color:#ABB2BF">);</span></span>
<span class="line"><span style="color:#61AFEF">            pushStream</span><span style="color:#ABB2BF">(</span><span style="color:#E06C75">answer</span><span style="color:#ABB2BF">);</span></span>
<span class="line"><span style="color:#ABB2BF">        }</span></span>
<span class="line"></span>
<span class="line"><span style="color:#7F848E;font-style:italic">        // 创建数据源</span></span>
<span class="line"><span style="color:#E5C07B">        window</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">onload</span><span style="color:#56B6C2"> =</span><span style="color:#C678DD"> async</span><span style="color:#ABB2BF"> () </span><span style="color:#C678DD">=></span><span style="color:#ABB2BF"> {</span></span>
<span class="line"><span style="color:#C678DD">            const</span><span style="color:#E5C07B"> localStream</span><span style="color:#56B6C2"> =</span><span style="color:#C678DD"> await</span><span style="color:#E5C07B"> navigator</span><span style="color:#ABB2BF">.</span><span style="color:#E5C07B">mediaDevices</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">getUserMedia</span><span style="color:#ABB2BF">({</span></span>
<span class="line"><span style="color:#E06C75">                video</span><span style="color:#ABB2BF">: </span><span style="color:#D19A66">true</span><span style="color:#ABB2BF">,</span></span>
<span class="line"><span style="color:#E06C75">                audio</span><span style="color:#ABB2BF">: </span><span style="color:#D19A66">true</span><span style="color:#ABB2BF">,</span></span>
<span class="line"><span style="color:#ABB2BF">            });</span></span>
<span class="line"><span style="color:#7F848E;font-style:italic">            // 显示数据源</span></span>
<span class="line"><span style="color:#E5C07B">            localVideo</span><span style="color:#ABB2BF">.</span><span style="color:#E06C75">srcObject</span><span style="color:#56B6C2"> =</span><span style="color:#E06C75"> localStream</span><span style="color:#ABB2BF">;</span></span>
<span class="line"><span style="color:#E5C07B">            localStream</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">getTracks</span><span style="color:#ABB2BF">().</span><span style="color:#61AFEF">forEach</span><span style="color:#ABB2BF">((</span><span style="color:#E06C75;font-style:italic">track</span><span style="color:#ABB2BF">) </span><span style="color:#C678DD">=></span><span style="color:#E5C07B"> pc1</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">addTrack</span><span style="color:#ABB2BF">(</span><span style="color:#E06C75">track</span><span style="color:#ABB2BF">, </span><span style="color:#E06C75">localStream</span><span style="color:#ABB2BF">));</span></span>
<span class="line"></span>
<span class="line"><span style="color:#C678DD">            const</span><span style="color:#E5C07B"> offer</span><span style="color:#56B6C2"> =</span><span style="color:#C678DD"> await</span><span style="color:#E5C07B"> pc1</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">createOffer</span><span style="color:#ABB2BF">();</span></span>
<span class="line"><span style="color:#E5C07B">            pc1</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">setLocalDescription</span><span style="color:#ABB2BF">(</span><span style="color:#E06C75">offer</span><span style="color:#ABB2BF">);</span></span>
<span class="line"><span style="color:#E5C07B">            console</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">warn</span><span style="color:#ABB2BF">(</span><span style="color:#98C379">'pc1 offer'</span><span style="color:#ABB2BF">, </span><span style="color:#E06C75">offer</span><span style="color:#ABB2BF">);</span></span>
<span class="line"><span style="color:#61AFEF">            pullStream</span><span style="color:#ABB2BF">(</span><span style="color:#E06C75">offer</span><span style="color:#ABB2BF">);</span></span>
<span class="line"><span style="color:#ABB2BF">        }</span></span>
<span class="line"><span style="color:#ABB2BF">    &#x3C;/</span><span style="color:#E06C75">script</span><span style="color:#ABB2BF">></span></span>
<span class="line"></span>
<span class="line"><span style="color:#ABB2BF">&#x3C;/</span><span style="color:#E06C75">body</span><span style="color:#ABB2BF">></span></span>
<span class="line"></span>
<span class="line"><span style="color:#ABB2BF">&#x3C;/</span><span style="color:#E06C75">html</span><span style="color:#ABB2BF">></span></span></code></pre>
<div class="line-numbers" aria-hidden="true" style="counter-reset:line-number 0"><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div></div><div class="collapsed-lines"></div></div><p><img src="@source/notes/每周一学/7.第七周学习/webrtc.webp" alt="webrtc"></p>
<p><strong>RTSP</strong></p>
<blockquote>
<p>what: 实时流式数据+控制协议</p>
</blockquote>
<blockquote>
<p>where: 视频监控,视频点播,交互式实时控制场景</p>
</blockquote>
<blockquote>
<p>why: 实时控制流的行为,如播放|停止|快进等操作</p>
</blockquote>
<p><strong>协议栈</strong></p>
<div class="language-cpp line-numbers-mode" data-highlighter="shiki" data-ext="cpp" style="background-color:#282c34;color:#abb2bf"><pre class="shiki one-dark-pro vp-code" v-pre=""><code><span class="line"><span style="color:#ABB2BF">应用层：控制指令（如播放、暂停、录制，通过RTSP协议本身定义）</span></span>
<span class="line"><span style="color:#ABB2BF">├─ 会话层：RTSP协议（基于TCP，负责建立</span><span style="color:#C678DD">/</span><span style="color:#ABB2BF">管理会话，交换控制指令）</span></span>
<span class="line"><span style="color:#ABB2BF">├─ 媒体传输层：</span></span>
<span class="line"><span style="color:#ABB2BF">│  ├─ 数据传输：RTP（音视频封装，基于UDP</span><span style="color:#C678DD">/</span><span style="color:#ABB2BF">TCP）</span></span>
<span class="line"><span style="color:#ABB2BF">│  └─ 媒体信息：RTCP（同WebRTC，监控丢包</span><span style="color:#C678DD">/</span><span style="color:#ABB2BF">延迟）</span></span>
<span class="line"><span style="color:#ABB2BF">├─ 传输层：</span></span>
<span class="line"><span style="color:#ABB2BF">│  ├─ 控制指令：TCP（可靠，确保指令不丢失）</span></span>
<span class="line"><span style="color:#ABB2BF">│  └─ 媒体数据：UDP（默认，低延迟）或TCP（丢包敏感时）</span></span>
<span class="line"><span style="color:#ABB2BF">└─ 编解码层：无强制规定（兼容H.</span><span style="color:#D19A66">264</span><span style="color:#ABB2BF">、H.</span><span style="color:#D19A66">265</span><span style="color:#ABB2BF">、MPEG</span><span style="color:#C678DD">-</span><span style="color:#D19A66">4</span><span style="color:#ABB2BF">等，由设备厂商定义）</span></span></code></pre>
<div class="line-numbers" aria-hidden="true" style="counter-reset:line-number 0"><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div></div></div><blockquote>
<p><a href="https://blog.csdn.net/m0_73443478/article/details/137231316" target="_blank" rel="noopener noreferrer">rtsp协议详解-最详细的流媒体传输协议-CSDN博客</a></p>
</blockquote>
<p><strong>RTMP</strong></p>
<blockquote>
<p>what: Adobe 的实时消息传输协议（服务器为中心）,早期用于 Flash 播放器的音视频传输，后成为直播 “推流到服务器” 的主流协议</p>
</blockquote>
<blockquote>
<p>where:  直播推流,内容分发,一般为单向传输</p>
</blockquote>
<blockquote>
<p>why: 底层基于TCP,传输可靠</p>
</blockquote>
<div class="language-cpp line-numbers-mode" data-highlighter="shiki" data-ext="cpp" style="background-color:#282c34;color:#abb2bf"><pre class="shiki one-dark-pro vp-code" v-pre=""><code><span class="line"><span style="color:#ABB2BF">应用层：推流</span><span style="color:#C678DD">/</span><span style="color:#ABB2BF">拉流逻辑（如主播推流到CDN）</span></span>
<span class="line"><span style="color:#ABB2BF">├─ 消息层：RTMP协议（封装音视频数据和控制消息）</span></span>
<span class="line"><span style="color:#ABB2BF">│  ├─ 控制消息：连接建立、带宽协商、流状态（播放</span><span style="color:#C678DD">/</span><span style="color:#ABB2BF">停止）</span></span>
<span class="line"><span style="color:#ABB2BF">│  └─ 媒体消息：音视频数据（基于FLV格式封装）</span></span>
<span class="line"><span style="color:#ABB2BF">├─ 传输层：TCP（必须，可靠传输，避免数据丢失）</span></span>
<span class="line"><span style="color:#ABB2BF">└─ 编解码层：固定兼容（视频：H.</span><span style="color:#D19A66">264</span><span style="color:#ABB2BF">为主；音频：AAC</span><span style="color:#C678DD">/</span><span style="color:#ABB2BF">MP3，由Flash生态限制）</span></span></code></pre>
<div class="line-numbers" aria-hidden="true" style="counter-reset:line-number 0"><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div></div></div><blockquote>
<p><a href="https://zhuanlan.zhihu.com/p/191542130" target="_blank" rel="noopener noreferrer">(1 封私信) 流媒体：RTMP 协议完全解析 - 知乎</a></p>
</blockquote>
<p><strong>主要区别</strong>:</p>
<ol>
<li>WebRTC一般用于点对点通信,RTSP/RTMP一般用于客户端与服务器之间通信</li>
<li>WebRTC和RTC都使用UDP+TCP等形式进行的数据流与控制流分离,而RTMP的数据流和控制流都使用TCP</li>
<li>延迟:WebRTC (100-300ms) &lt; RTSP(100-500ms) &lt; RTMP (300-1000ms) &lt; HTTP</li>
<li>RTMP数据流一般是flv,f4v,RTSP数据流一般是ts,mp4</li>
</ol>
<blockquote>
<p><a href="https://www.hangge.com/blog/cache/detail_1325.html" target="_blank" rel="noopener noreferrer">RTMP、RTSP、HTTP视频协议详解（附：直播流地址、播放软件）</a></p>
</blockquote>
<h3 id="以上三个协议控制流和数据流是如何分离的-又是如何协同工作的" tabindex="-1"><a class="header-anchor" href="#以上三个协议控制流和数据流是如何分离的-又是如何协同工作的"><span>以上三个协议控制流和数据流是如何分离的，又是如何协同工作的?</span></a></h3>
<p>使用多个轨道进行数据流,控制流,信息流分离,WebRTC和RTSP都是使用TCP进行控制流的传输,使用UDP进行的数据流传输,</p>
<p>而RTMP为了信息传递的可靠,控制流和信息流都使用TCP进行传输.</p>
<p>一般对于观看一个视频而言,视频的每一帧画面通过数据流进行显示,信息流一般表示视频当前的码率,帧率,分辨率等信息,控制流一般控制视频的播放,快进,停止等操作.</p>
<h3 id="扩展-用你现有的知识-绘制一个推流服务的结构图-考虑尽可能低拷贝与数据cdn分发" tabindex="-1"><a class="header-anchor" href="#扩展-用你现有的知识-绘制一个推流服务的结构图-考虑尽可能低拷贝与数据cdn分发"><span>扩展：用你现有的知识，绘制一个推流服务的结构图，考虑尽可能低拷贝与数据CDN分发</span></a></h3>
<p><img src="@source/notes/每周一学/7.第七周学习/cdn-stream.png" alt=""></p>
<h3 id="为什么流媒体通讯过程要求尽可能点到点通信-点到点通讯的协议栈又长什么样" tabindex="-1"><a class="header-anchor" href="#为什么流媒体通讯过程要求尽可能点到点通信-点到点通讯的协议栈又长什么样"><span>为什么流媒体通讯过程要求尽可能点到点通信？点到点通讯的协议栈又长什么样？</span></a></h3>
<ol>
<li>减少数据传输路径长度,即数据拷贝次数</li>
<li>降低服务器带宽压力,说白了就省钱</li>
</ol>
<div class="language-cpp line-numbers-mode" data-highlighter="shiki" data-ext="cpp" style="background-color:#282c34;color:#abb2bf"><pre class="shiki one-dark-pro vp-code" v-pre=""><code><span class="line"><span style="color:#ABB2BF">应用层：WebRTC</span></span>
<span class="line"><span style="color:#ABB2BF">↓</span></span>
<span class="line"><span style="color:#ABB2BF">媒体控制层：SDP（会话描述，约定编码格式等）、RTCP（媒体质量反馈，比如“网络卡了”）</span></span>
<span class="line"><span style="color:#ABB2BF">↓</span></span>
<span class="line"><span style="color:#ABB2BF">媒体传输层：RTP（实时传输协议，包装视频</span><span style="color:#C678DD">/</span><span style="color:#ABB2BF">音频数据）</span></span>
<span class="line"><span style="color:#ABB2BF">↓</span></span>
<span class="line"><span style="color:#ABB2BF">网络穿透层：ICE（找到P2P路径）、STUN（获取公网地址）、TURN（穿透失败时中转，备用）</span></span>
<span class="line"><span style="color:#ABB2BF">↓</span></span>
<span class="line"><span style="color:#ABB2BF">传输层：UDP（数据流）</span><span style="color:#C678DD">/</span><span style="color:#ABB2BF"> TCP（控制流）</span></span></code></pre>
<div class="line-numbers" aria-hidden="true" style="counter-reset:line-number 0"><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div></div></div><h3 id="系统学习以下协议nat、turn、stun、ice-绘制出nat穿透过程的时序图" tabindex="-1"><a class="header-anchor" href="#系统学习以下协议nat、turn、stun、ice-绘制出nat穿透过程的时序图"><span>系统学习以下协议NAT、TURN、STUN、ICE,绘制出NAT穿透过程的时序图</span></a></h3>
<p>NAT: 网络地址转换协议</p>
<p>ICE = NAT + TURN+ STUN</p>
<p><img src="@source/notes/每周一学/7.第七周学习/ice.png" alt=""></p>
<p><img src="@source/notes/每周一学/7.第七周学习/nat-0.png" alt=""></p>
<p><img src="@source/notes/每周一学/7.第七周学习/nat-1.png" alt=""></p>
<p><img src="@source/notes/每周一学/7.第七周学习/nat-2.png" alt=""></p>
<p><img src="@source/notes/每周一学/7.第七周学习/nat-3.png" alt=""></p>
<p><img src="@source/notes/每周一学/7.第七周学习/nat-4.png" alt=""></p>
<p><em>举例解释:用「国际快递」比喻 ICE 的工作流程</em></p>
<p><em>角色对应：</em></p>
<ul>
<li><strong>Alice 和 Bob</strong>：两台想通信的设备（如视频会议的两端）。</li>
<li><strong>家庭地址</strong>：设备的私有 IP（如 192.168.1.100，只有家里人知道）。</li>
<li><strong>小区门卫室</strong>：NAT 设备（家庭路由器，给每个设备分配私有 IP）。</li>
<li><strong>小区地址</strong>：公网 IP（如 203.0.113.45，外人能看到的小区大门地址）。</li>
<li><strong>国际邮局</strong>：STUN/TURN 服务器（帮你找到小区地址或中转信件）。</li>
</ul>
<p><em>步骤分解</em>：</p>
<ol>
<li><strong>发现自己的地址</strong>（STUN 阶段）：
Alice 问国际邮局：“我家小区的公开地址是什么？”
邮局回复：“你的小区地址是 203.0.113.45，门卫室会把寄到这个地址的信转给你。”
→ 这就是<strong>STUN 服务器</strong>的作用：让设备知道自己的公网 IP。</li>
<li><strong>尝试直接投递</strong>（打洞阶段）：
Alice 和 Bob 都知道了自己的小区地址，于是互相尝试直接寄信：
<ul>
<li>Alice → Bob：“我从 203.0.113.45 给你寄信，门卫室能收到吗？”</li>
<li>Bob → Alice：“我从 198.51.100.78 给你寄信，你的门卫室能收到吗？”
→ 如果双方的门卫室允许这种「试探性投递」（称为<strong>打洞</strong>），信件就能直接送达，这就是<strong>P2P 连接成功</strong>。</li>
</ul>
</li>
<li><strong>中转方案</strong>（TURN 阶段）：
如果直接投递失败（比如 Bob 的小区门卫不让陌生信件进入），两人约定：
“都把信寄到国际邮局（TURN 服务器），由邮局转发给对方。”
→ 这就是<strong>TURN 服务器</strong>的作用：当 P2P 失败时，作为中间人转发数据。</li>
</ol>
<blockquote>
<p><a href="https://blog.csdn.net/qq_64162562/article/details/136732995" target="_blank" rel="noopener noreferrer">NAT基本原理及穿透详解(打洞)_nat穿透-CSDN博客</a></p>
<p><a href="https://www.cnblogs.com/mlgjb/p/8243690.html" target="_blank" rel="noopener noreferrer">P2P技术详解(三)：P2P技术之STUN、TURN、ICE详解 - 哦摩西罗伊 - 博客园</a></p>
<p><a href="https://zhuanlan.zhihu.com/p/1890724663333736762" target="_blank" rel="noopener noreferrer"> p2p通信原理——STUN 协议详解 - 知乎</a></p>
</blockquote>
<h2 id="知识扩展" tabindex="-1"><a class="header-anchor" href="#知识扩展"><span>知识扩展</span></a></h2>
<h3 id="从图像到视频是如何压缩编码的" tabindex="-1"><a class="header-anchor" href="#从图像到视频是如何压缩编码的"><span>从图像到视频是如何压缩编码的？</span></a></h3>
<p>一般是从两个方向考虑一个是帧内压缩,另一个是帧间压缩</p>
<p><em>帧内压缩</em></p>
<p>将每幅图片分割为8X8或16X16的宏块,然后将RGBA(红绿蓝透明度)转成YUV(Y表示亮度,U和V表示色度,用于描述色相和饱和度,色彩饱和度也就是颜色的深浅的意思),由原来的三,四个字节表示一个像素转为1(Y)+0.5(U)+0.5(V)也就是2个字节.</p>
<p><em>帧间压缩</em></p>
<p>将每一帧分为三种类型</p>
<ul>
<li><strong>I帧</strong>:关键帧，完整编码的图像（类似 JPEG，不依赖其他帧）</li>
<li><strong>P帧</strong>: 预测帧,只记录与前一个 I/P 帧的 “差异”（比如嘴唇动了，背景复用）</li>
<li><strong>B 帧</strong>:双向预测帧，根据前后帧预测当前帧（比如预测物体运动轨迹）</li>
</ul>
<p><em>视频编码标准演进：从 H.264 到 AV1</em></p>
<p>不同标准的核心差异在于<strong>压缩效率</strong>（同等画质下的数据量）和<strong>计算复杂度</strong>（编码 / 解码所需算力）。</p>
<table>
<thead>
<tr>
<th>标准</th>
<th>推出时间</th>
<th>代表应用</th>
<th>压缩效率（对比 H.264）</th>
<th>特点</th>
</tr>
</thead>
<tbody>
<tr>
<td>H.264</td>
<td>2003 年</td>
<td>YouTube、直播</td>
<td>基准线</td>
<td>兼容性最好，至今仍广泛使用</td>
</tr>
<tr>
<td>H.265</td>
<td>2013 年</td>
<td>4K/8K 视频、流媒体</td>
<td>提升 50%</td>
<td>更高压缩率，但解码成本高</td>
</tr>
<tr>
<td>VP9</td>
<td>2013 年</td>
<td>YouTube（WebM 格式）</td>
<td>类似 H.265</td>
<td>开源，免专利费</td>
</tr>
<tr>
<td>AV1</td>
<td>2018 年</td>
<td>Netflix、Twitch</td>
<td>提升 30%（对比 H.265）</td>
<td>开源，专为 8K / 高动态范围设计</td>
</tr>
<tr>
<td>VVC</td>
<td>2020 年</td>
<td>未来 8K 广播、视频会议</td>
<td>比 H.265 再提升 50%</td>
<td>压缩率逆天，但计算复杂度极高</td>
</tr>
</tbody>
</table>
<blockquote>
<p><a href="https://zhuanlan.zhihu.com/p/601614313" target="_blank" rel="noopener noreferrer"> JPEG图像压缩详解和代码实现 - 知乎</a></p>
<p><a href="https://blog.csdn.net/linchare/article/details/100725648" target="_blank" rel="noopener noreferrer">JPEG压缩算法详解-CSDN博客</a></p>
<p><a href="https://blog.csdn.net/Oh_my_God_L_C/article/details/111290382" target="_blank" rel="noopener noreferrer">YUV444,YUV422,YUV420中的4,2,0代表什么意思?_ycbcr420 中0是什么意思-CSDN博客</a></p>
<p><a href="https://www.cnblogs.com/lfri/p/10426113.html" target="_blank" rel="noopener noreferrer">RGB颜色空间、色调、饱和度、亮度，HSV颜色空间详解 - Rogn - 博客园</a></p>
<p><a href="https://zhuanlan.zhihu.com/p/113122344" target="_blank" rel="noopener noreferrer"> 一文读懂 YUV 的采样与格式 - 知乎</a></p>
</blockquote>
<h3 id="bitmap、yuv格式的图像分别是如何表达像素的-rgb和bgr有什么区别" tabindex="-1"><a class="header-anchor" href="#bitmap、yuv格式的图像分别是如何表达像素的-rgb和bgr有什么区别"><span>BITMAP、YUV格式的图像分别是如何表达像素的？RGB和BGR有什么区别？</span></a></h3>
<p>BITMAP一般使用RGB表示一个像素(0-255)</p>
<p>YUV-亮度-色度来表示一个像素</p>
<p>RGB和BGR区别在于在文件字节顺序不一样,RGB是Red-Green-Blue的顺序,BGR是Blue-Green-Red的顺序</p>
<h3 id="png-jepg是如何压缩原图片的" tabindex="-1"><a class="header-anchor" href="#png-jepg是如何压缩原图片的"><span>PNG，JEPG是如何压缩原图片的？</span></a></h3>
<table>
<thead>
<tr>
<th>维度</th>
<th>JPEG</th>
<th>PNG</th>
</tr>
</thead>
<tbody>
<tr>
<td>压缩类型</td>
<td>有损（丢弃高频和色度细节）</td>
<td>无损（不丢任何信息）</td>
</tr>
<tr>
<td>核心手段</td>
<td>YUV 下采样 + DCT 量化 + 熵编码</td>
<td>索引色 / 滤波 + DEFLATE 编码</td>
</tr>
<tr>
<td>体积</td>
<td>小（适合高压缩率）</td>
<td>大（无损导致）</td>
</tr>
<tr>
<td>画质特点</td>
<td>可能模糊、出现方块效应</td>
<td>完全保留原始画质</td>
</tr>
<tr>
<td>适用场景</td>
<td>照片、色彩丰富的自然图像</td>
<td>图标、文字、透明图、图形</td>
</tr>
<tr>
<td>透明支持</td>
<td>不支持（需扩展格式如 JPEG 2000）</td>
<td>支持 α 透明通道</td>
</tr>
</tbody>
</table>
<blockquote>
<p><a href="https://zhuanlan.zhihu.com/p/601614313" target="_blank" rel="noopener noreferrer"> JPEG图像压缩详解和代码实现 - 知乎</a></p>
</blockquote>
<h3 id="视频编码算法比起图片压缩编码多了哪些要处理的东西" tabindex="-1"><a class="header-anchor" href="#视频编码算法比起图片压缩编码多了哪些要处理的东西"><span>视频编码算法比起图片压缩编码多了哪些要处理的东西？</span></a></h3>
<p>时间冗余处理（运动估计 / 补偿） + 帧间依赖管理 + 动态码率控制</p>
<h3 id="学习了解mpeg-h264编码算法" tabindex="-1"><a class="header-anchor" href="#学习了解mpeg-h264编码算法"><span>学习了解MPEG，H264编码算法</span></a></h3>
<p><a href="https://cloud.tencent.com/developer/article/2270869" target="_blank" rel="noopener noreferrer">视频编码简史:H.263/H.264/H.265和MPEG2/MPEG4以及VP9，H264优点-腾讯云开发者社区-腾讯云</a></p>
<h3 id="mp4是什么" tabindex="-1"><a class="header-anchor" href="#mp4是什么"><span>MP4是什么？</span></a></h3>
<p>MP4 是一种<strong>多媒体容器格式</strong>,<strong>将视频、音频、字幕、元数据等多种 “流数据” 封装到一个文件中</strong>,不是编码格式</p>
<h3 id="了解h265-h266-av1-vp9算法-了解即可-别投入精力学习" tabindex="-1"><a class="header-anchor" href="#了解h265-h266-av1-vp9算法-了解即可-别投入精力学习"><span>了解H265,H266,AV1,VP9算法（了解即可，别投入精力学习）</span></a></h3>
<blockquote>
<p><a href="https://blog.csdn.net/jianluoffice/article/details/148693813" target="_blank" rel="noopener noreferrer">视频编码怎么选？H.264、H.265、VP9、AV1全解析_h.264和av1-CSDN博客</a></p>
</blockquote>
<h2 id="练习" tabindex="-1"><a class="header-anchor" href="#练习"><span>练习</span></a></h2>
<p>由于搭建信令服务器，ICE服务较为麻烦，我们使用等价的练习方法进行推流练习（B站的实现方法)</p>
<p>要求：编写SSE接口，使用视频文件名调用，编码任意，将java服务端的视频按照每三到五个I帧极其相关B帧与P帧为一组报文，SSE推给前端 (前端作业：使用video组件播放视频流)</p>
<h2 id="进阶思考与练习" tabindex="-1"><a class="header-anchor" href="#进阶思考与练习"><span>进阶思考与练习：</span></a></h2>
<p>如果是预先播放视频的话，我能不能将视频编码片段直接作为nginx可访问到的文件，从java控制文件的读取顺序即可？尝试实现一下。</p>
<blockquote>
<p>回头在研究研究把</p>
</blockquote>
<p><img src="@source/notes/每周一学/7.第七周学习/mse.png" alt=""></p>
<blockquote>
<p><a href="https://segmentfault.com/a/1190000044568143" target="_blank" rel="noopener noreferrer">这可能是最详细的javaCV-FFmpeg防踩坑入门了 - 编程苦旅 - SegmentFault 思否</a></p>
<p><a href="https://zhuanlan.zhihu.com/p/22000376913" target="_blank" rel="noopener noreferrer">Web 音视频（二）在浏览器中解析视频 - 知乎</a></p>
<p><a href="https://blog.csdn.net/qq_44895902/article/details/135387246" target="_blank" rel="noopener noreferrer">基于WebAssembly无插件解码H264/H265码流播放器_webassembly解决h256-CSDN博客</a></p>
<p><a href="https://www.zhangxinxu.com/wordpress/2021/03/ffmpeg-wasm-audio-video-merge/" target="_blank" rel="noopener noreferrer">借助ffmpeg.wasm纯前端实现多音频和视频的合成</a></p>
<p><a href="https://zhuanlan.zhihu.com/p/632677093" target="_blank" rel="noopener noreferrer">( FFmpeg功能命令集合(超详细) - 知乎</a></p>
<p><a href="https://wklchris.github.io/blog/FFmpeg/index.html" target="_blank" rel="noopener noreferrer">FFmpeg 教程 — FFmpeg 教程 文档</a></p>
<p><a href="https://ffmpeg.github.net.cn/documentation.html" target="_blank" rel="noopener noreferrer">文档_ FFmpeg中文网</a><a href="https://zhuanlan.zhihu.com/p/622152133" target="_blank" rel="noopener noreferrer"> 超详细的h264码流结构解析！ - 知乎</a></p>
<p><a href="https://blog.csdn.net/zhying719/article/details/123725415" target="_blank" rel="noopener noreferrer">FFmpeg 中 AVPacket 与 AVFrame 中数据的传递与释放-CSDN博客</a></p>
<p><a href="https://blog.csdn.net/u013684730/article/details/73108670" target="_blank" rel="noopener noreferrer">FFmpeg中AVPacket和AVFrame关系_avpacket frame-CSDN博客</a></p>
<p><a href="https://blog.csdn.net/weixin_44322983/article/details/124310620" target="_blank" rel="noopener noreferrer">AVPacket，AVFrame，AVStream结构体_avstream 和 aframe-CSDN博客</a></p>
<p><a href="https://zhuanlan.zhihu.com/p/556657311" target="_blank" rel="noopener noreferrer"> AVPacket，AVFrame，AVStream结构体 - 知乎</a></p>
<p><a href="https://www.cnblogs.com/lidabo/p/15412094.html" target="_blank" rel="noopener noreferrer">ffmpeg内存模型及AVPacket和AVFrame API基本使用 - DoubleLi - 博客园</a></p>
<p><a href="https://blog.csdn.net/youjin/article/details/125093820" target="_blank" rel="noopener noreferrer">JavaCV入门指南：帧录制器/推流器(FrameRecorder)的原理与应用_ffmpegframerecorder-CSDN博客</a></p>
<p><a href="https://blog.csdn.net/boling_cavalry/article/details/121434969" target="_blank" rel="noopener noreferrer">JavaCV推流实战(MP4文件)_javacv 推流本地mp4-CSDN博客</a></p>
<p><a href="https://blog.csdn.net/shulianghan/article/details/143785665" target="_blank" rel="noopener noreferrer">【FFmpeg】FFmpeg 内存结构 ① ( AVPacket 队列 和 AVFrame 队列 | AVPacket 数据的 深拷贝 和 浅拷贝 | AVPacket 的 引用计数器机制 )_linux c链表队列放入ffmpeg的avpacket-CSDN博客</a></p>
<p><a href="https://blog.51cto.com/u_16213437/8757708" target="_blank" rel="noopener noreferrer">javacv frame to AvPacket_mob64ca12f1c6f8的技术博客_51CTO博客</a></p>
<p><a href="https://blog.51cto.com/u_14202100/12581980" target="_blank" rel="noopener noreferrer">【FFmpeg】FFmpeg 内存结构 ① ( AVPacket 队列 和 AVFrame 队列 | AVPacket 数据的 深拷贝 和 浅拷贝 | AVPacket 的 引用计数器机制 )_51CTO博客_ffmpeg avfoundation</a></p>
<p><a href="https://blog.csdn.net/eguid_1/article/details/82875343" target="_blank" rel="noopener noreferrer">javaCV入门指南：序章_javacv中文手册-CSDN博客</a><a href="https://www.lewinblog.com/blog/page/2022/221028-ffmpeg-wasm.md" target="_blank" rel="noopener noreferrer">ffmpeg.wasm 踩坑体验 - Lewin's Blog</a></p>
</blockquote>
<h2 id="最终解决方案" tabindex="-1"><a class="header-anchor" href="#最终解决方案"><span>最终解决方案</span></a></h2>
<p>经过踩了一个多月的坑,决定使用B站的流媒体协议,<strong>DASH</strong></p>
<p>即会对一个视频进行预处理,生成一个mpd文件和一系列m4s文件,m4s文件即fmp4文件(也就是一个moov盒子 + 一堆moof + mdat),其中moof是分片的标志标志位</p>
<div class="language-java line-numbers-mode has-collapsed-lines collapsed" data-highlighter="shiki" data-ext="java" style="--vp-collapsed-lines:15;background-color:#282c34;color:#abb2bf"><pre class="shiki one-dark-pro vp-code" v-pre=""><code><span class="line"><span style="color:#ABB2BF">@</span><span style="color:#E5C07B">CrossOrigin</span></span>
<span class="line"><span style="color:#ABB2BF">@</span><span style="color:#E5C07B">RestController</span></span>
<span class="line"><span style="color:#ABB2BF">@</span><span style="color:#E5C07B">RequestMapping</span><span style="color:#E06C75">(</span><span style="color:#98C379">"/dash"</span><span style="color:#E06C75">)</span></span>
<span class="line"><span style="color:#C678DD">public</span><span style="color:#C678DD"> class</span><span style="color:#E5C07B"> DashController</span><span style="color:#ABB2BF"> {</span></span>
<span class="line"></span>
<span class="line"><span style="color:#C678DD">    private</span><span style="color:#C678DD"> final</span><span style="color:#E5C07B"> String</span><span style="color:#E06C75"> path </span><span style="color:#56B6C2">=</span><span style="color:#98C379"> "C:/tmp/"</span><span style="color:#ABB2BF">;</span></span>
<span class="line"></span>
<span class="line"><span style="color:#ABB2BF">    @</span><span style="color:#E5C07B">GetMapping</span><span style="color:#E06C75">(</span><span style="color:#98C379">"/video/{videoId}"</span><span style="color:#E06C75">)</span></span>
<span class="line"><span style="color:#C678DD">    public</span><span style="color:#E5C07B"> String</span><span style="color:#61AFEF"> getVideo</span><span style="color:#ABB2BF">(@</span><span style="color:#E5C07B">PathVariable</span><span style="color:#E5C07B"> String</span><span style="color:#E06C75;font-style:italic"> videoId</span><span style="color:#ABB2BF">)</span><span style="color:#ABB2BF"> {</span></span>
<span class="line"><span style="color:#E5C07B">        String</span><span style="color:#E06C75"> in</span><span style="color:#56B6C2"> =</span><span style="color:#ABB2BF"> path </span><span style="color:#56B6C2">+</span><span style="color:#ABB2BF"> videoId </span><span style="color:#56B6C2">+</span><span style="color:#98C379"> "/"</span><span style="color:#56B6C2"> +</span><span style="color:#ABB2BF"> videoId </span><span style="color:#56B6C2">+</span><span style="color:#98C379"> ".mp4"</span><span style="color:#ABB2BF">;</span></span>
<span class="line"><span style="color:#E5C07B">        String</span><span style="color:#E06C75"> out</span><span style="color:#56B6C2"> =</span><span style="color:#ABB2BF"> path </span><span style="color:#56B6C2">+</span><span style="color:#ABB2BF"> videoId </span><span style="color:#56B6C2">+</span><span style="color:#98C379"> "/"</span><span style="color:#56B6C2"> +</span><span style="color:#ABB2BF"> videoId </span><span style="color:#56B6C2">+</span><span style="color:#98C379"> ".mpd"</span><span style="color:#ABB2BF">;</span></span>
<span class="line"><span style="color:#E5C07B">        File</span><span style="color:#E06C75"> file</span><span style="color:#56B6C2"> =</span><span style="color:#C678DD"> new</span><span style="color:#61AFEF"> File</span><span style="color:#ABB2BF">(out);</span></span>
<span class="line"><span style="color:#C678DD">        if</span><span style="color:#ABB2BF">(</span><span style="color:#56B6C2">!</span><span style="color:#E5C07B">file</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">exists</span><span style="color:#ABB2BF">()) </span><span style="color:#E5C07B">FFmpegUtil</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">dash</span><span style="color:#ABB2BF">(in,out); </span><span style="color:#7F848E;font-style:italic">// 如果mpd文件存在,就直接返回mpd路径,否则就调用ffmpeg进行切片,在返回mpd文件路径</span></span>
<span class="line"><span style="color:#C678DD">        return</span><span style="color:#ABB2BF"> videoId </span><span style="color:#56B6C2">+</span><span style="color:#98C379"> "/"</span><span style="color:#56B6C2"> +</span><span style="color:#ABB2BF"> videoId </span><span style="color:#56B6C2">+</span><span style="color:#98C379"> ".mpd"</span><span style="color:#ABB2BF">;</span></span>
<span class="line"><span style="color:#ABB2BF">    }</span></span>
<span class="line"></span>
<span class="line"><span style="color:#ABB2BF">}</span></span></code></pre>
<div class="line-numbers" aria-hidden="true" style="counter-reset:line-number 0"><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div></div><div class="collapsed-lines"></div></div><div class="language-java line-numbers-mode" data-highlighter="shiki" data-ext="java" style="background-color:#282c34;color:#abb2bf"><pre class="shiki one-dark-pro vp-code" v-pre=""><code><span class="line"><span style="color:#C678DD">public</span><span style="color:#C678DD"> class</span><span style="color:#E5C07B"> FFmpegUtil</span><span style="color:#ABB2BF"> {</span></span>
<span class="line"></span>
<span class="line"><span style="color:#C678DD">    public</span><span style="color:#C678DD"> static</span><span style="color:#E5C07B"> String</span><span style="color:#61AFEF"> dash</span><span style="color:#ABB2BF">(</span><span style="color:#E5C07B">String</span><span style="color:#E06C75;font-style:italic"> in</span><span style="color:#ABB2BF">, </span><span style="color:#E5C07B">String</span><span style="color:#E06C75;font-style:italic"> out</span><span style="color:#ABB2BF">)</span><span style="color:#ABB2BF"> {</span></span>
<span class="line"><span style="color:#E5C07B">        String</span><span style="color:#E06C75"> cmd</span><span style="color:#56B6C2"> =</span><span style="color:#98C379"> "ffmpeg -i "</span><span style="color:#56B6C2"> +</span><span style="color:#ABB2BF"> in </span><span style="color:#56B6C2">+</span><span style="color:#98C379"> " -c:v copy -c:a copy -map 0 -f dash -seg_duration 2 "</span><span style="color:#56B6C2"> +</span><span style="color:#ABB2BF"> out;</span></span>
<span class="line"><span style="color:#C678DD">        try</span><span style="color:#ABB2BF"> {</span></span>
<span class="line"><span style="color:#E5C07B">            Process</span><span style="color:#E06C75"> process</span><span style="color:#56B6C2"> =</span><span style="color:#C678DD"> new</span><span style="color:#61AFEF"> ProcessBuilder</span><span style="color:#ABB2BF">(cmd).</span><span style="color:#61AFEF">start</span><span style="color:#ABB2BF">();</span></span>
<span class="line"><span style="color:#E5C07B">            process</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">waitFor</span><span style="color:#ABB2BF">();</span></span>
<span class="line"><span style="color:#ABB2BF">        } </span><span style="color:#C678DD">catch</span><span style="color:#ABB2BF"> (</span><span style="color:#E5C07B">IOException</span><span style="color:#ABB2BF"> | </span><span style="color:#E5C07B">InterruptedException</span><span style="color:#E06C75;font-style:italic"> e</span><span style="color:#ABB2BF">) {</span></span>
<span class="line"><span style="color:#E5C07B">            e</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">printStackTrace</span><span style="color:#ABB2BF">();</span></span>
<span class="line"><span style="color:#ABB2BF">        }</span></span>
<span class="line"><span style="color:#C678DD">        return</span><span style="color:#ABB2BF"> out;</span></span>
<span class="line"><span style="color:#ABB2BF">    }</span></span>
<span class="line"><span style="color:#ABB2BF">}</span></span></code></pre>
<div class="line-numbers" aria-hidden="true" style="counter-reset:line-number 0"><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div></div></div><div class="language-html line-numbers-mode has-collapsed-lines collapsed" data-highlighter="shiki" data-ext="html" style="--vp-collapsed-lines:15;background-color:#282c34;color:#abb2bf"><pre class="shiki one-dark-pro vp-code" v-pre=""><code><span class="line"><span style="color:#ABB2BF">&#x3C;!</span><span style="color:#E06C75">DOCTYPE</span><span style="color:#D19A66"> html</span><span style="color:#ABB2BF">></span></span>
<span class="line"><span style="color:#ABB2BF">&#x3C;</span><span style="color:#E06C75">html</span><span style="color:#D19A66"> lang</span><span style="color:#ABB2BF">=</span><span style="color:#98C379">"en"</span><span style="color:#ABB2BF">></span></span>
<span class="line"><span style="color:#ABB2BF">&#x3C;</span><span style="color:#E06C75">head</span><span style="color:#ABB2BF">></span></span>
<span class="line"><span style="color:#ABB2BF">    &#x3C;</span><span style="color:#E06C75">meta</span><span style="color:#D19A66"> charset</span><span style="color:#ABB2BF">=</span><span style="color:#98C379">"UTF-8"</span><span style="color:#ABB2BF">></span></span>
<span class="line"><span style="color:#ABB2BF">    &#x3C;</span><span style="color:#E06C75">meta</span><span style="color:#D19A66"> name</span><span style="color:#ABB2BF">=</span><span style="color:#98C379">"viewport"</span><span style="color:#D19A66"> content</span><span style="color:#ABB2BF">=</span><span style="color:#98C379">"width=device-width, initial-scale=1.0"</span><span style="color:#ABB2BF">></span></span>
<span class="line"><span style="color:#ABB2BF">    &#x3C;</span><span style="color:#E06C75">title</span><span style="color:#ABB2BF">>Dash&#x3C;/</span><span style="color:#E06C75">title</span><span style="color:#ABB2BF">></span></span>
<span class="line"><span style="color:#ABB2BF">    &#x3C;</span><span style="color:#E06C75">script</span><span style="color:#D19A66"> src</span><span style="color:#ABB2BF">=</span><span style="color:#98C379">"https://cdn.jsdelivr.net/npm/dashjs/dist/dash.all.min.js"</span><span style="color:#ABB2BF">>&#x3C;/</span><span style="color:#E06C75">script</span><span style="color:#ABB2BF">></span></span>
<span class="line"><span style="color:#ABB2BF">&#x3C;/</span><span style="color:#E06C75">head</span><span style="color:#ABB2BF">></span></span>
<span class="line"><span style="color:#ABB2BF">&#x3C;</span><span style="color:#E06C75">body</span><span style="color:#ABB2BF">></span></span>
<span class="line"><span style="color:#ABB2BF">    &#x3C;</span><span style="color:#E06C75">video</span><span style="color:#D19A66"> id</span><span style="color:#ABB2BF">=</span><span style="color:#98C379">"videoPlayer"</span><span style="color:#D19A66"> controls</span><span style="color:#ABB2BF">>&#x3C;/</span><span style="color:#E06C75">video</span><span style="color:#ABB2BF">></span></span>
<span class="line"><span style="color:#ABB2BF">    &#x3C;</span><span style="color:#E06C75">script</span><span style="color:#ABB2BF"> ></span></span>
<span class="line"><span style="color:#ABB2BF">        (</span><span style="color:#C678DD">async</span><span style="color:#ABB2BF"> () </span><span style="color:#C678DD">=></span><span style="color:#ABB2BF"> {</span></span>
<span class="line"><span style="color:#C678DD">            var</span><span style="color:#E06C75"> player</span><span style="color:#56B6C2"> =</span><span style="color:#E5C07B"> dashjs</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">MediaPlayer</span><span style="color:#ABB2BF">().</span><span style="color:#61AFEF">create</span><span style="color:#ABB2BF">(); </span><span style="color:#7F848E;font-style:italic">// 使用dash.js播放dash</span></span>
<span class="line"><span style="color:#C678DD">            let</span><span style="color:#E06C75"> url</span><span style="color:#56B6C2">  =</span><span style="color:#C678DD"> await</span><span style="color:#61AFEF"> fetch</span><span style="color:#ABB2BF">(</span><span style="color:#98C379">"http://localhost:13702/dash/video/demo"</span><span style="color:#ABB2BF">);</span></span>
<span class="line"><span style="color:#E06C75">            url</span><span style="color:#56B6C2"> =</span><span style="color:#C678DD"> await</span><span style="color:#E5C07B"> url</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">text</span><span style="color:#ABB2BF">();</span></span>
<span class="line"><span style="color:#E5C07B">            player</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">initialize</span><span style="color:#ABB2BF">(</span><span style="color:#E5C07B">document</span><span style="color:#ABB2BF">.</span><span style="color:#61AFEF">getElementById</span><span style="color:#ABB2BF">(</span><span style="color:#98C379">"videoPlayer"</span><span style="color:#ABB2BF">), </span><span style="color:#98C379">"http://localhost:13702/"</span><span style="color:#56B6C2"> +</span><span style="color:#E06C75"> url</span><span style="color:#ABB2BF">, </span><span style="color:#D19A66">false</span><span style="color:#ABB2BF">);</span></span>
<span class="line"><span style="color:#ABB2BF">        })();</span></span>
<span class="line"></span>
<span class="line"><span style="color:#ABB2BF">    &#x3C;/</span><span style="color:#E06C75">script</span><span style="color:#ABB2BF">></span></span>
<span class="line"><span style="color:#ABB2BF">&#x3C;/</span><span style="color:#E06C75">body</span><span style="color:#ABB2BF">></span></span>
<span class="line"><span style="color:#ABB2BF">&#x3C;/</span><span style="color:#E06C75">html</span><span style="color:#ABB2BF">></span></span></code></pre>
<div class="line-numbers" aria-hidden="true" style="counter-reset:line-number 0"><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div><div class="line-number"></div></div><div class="collapsed-lines"></div></div><blockquote>
<p><a href="https://www.cnblogs.com/saryli/p/8963006.html" target="_blank" rel="noopener noreferrer">DASH简介及使用方法(FFmpeg, MP4Box) - 清明-心若淡定 - 博客园</a></p>
<p><a href="https://zhuanlan.zhihu.com/p/4972879441" target="_blank" rel="noopener noreferrer">M4S格式介绍及对B站缓存文件方案的思考 - 知乎</a></p>
</blockquote>
</div></template>


